Search for: All records

Using natural-language feedback to guide image generation and manipulation can greatly lower the required efforts and skills. This topic has received increased attention in recent years through refinement of Generative Adversarial Networks (GANs); however, most existing works are limited to single-round interaction, which is not reflective of real world interactive image editing workflows. Furthermore, previous works dealing with multi-round scenarios are limited to predefined feedback sequences, which is also impractical. In this paper, we propose a novel framework for Text-based Interactive image generation and manipulation (TiGAN) that responds to users' natural-language feedback. TiGAN utilizes the powerful pre-trained CLIP model to understand users' natural-language feedback and exploits contrastive learning for a better text-to-image mapping. To maintain the image consistency during interactions, TiGAN generates intermediate feature vectors aligned with the feedback and selectively feeds these vectors to our proposed generative model. Empirical results on several datasets show that TiGAN improves both interaction efficiency and image quality while better avoids undesirable image manipulation during interactions. 

An important hypothesis in animal cell biology is that an animal’s acute exercise regimen affects some subcellular structures, for example mitochondrial morphology, in its muscle tissue. This paper investigates that hypothesis using a nonparametric metric-based energy test for comparing mitochondrial populations. It explores two shape spaces—the elastic shape space and Kendall shape space—and five corresponding shape metrics on these spaces. The results overwhelmingly point to the statistical significance of the effect of an acute exercise regimen on the shape of SS-type mitochondria. Although past studies based on specific morphological features derived from mitochondria failed to detect this significance. In this analysis, a potentially significant factor is cell membership and a k-sample generalization of the energy test—the DISCO test shows that the cell effect is indeed significant. The energy test cannot be applied directly due to the hierarchical structure of the distance matrix. We propose a compression method to remove the significant cell effect while testing for the exercise effect. With this compression, only the elastic scaled metric shows statistical significance of the exercise factor in this more complicated scenario. This result is because the elastic-scaled metric is more sensitive to subtle changes in mitochondrial shapes caused by acute exercise.